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■OREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI)  is  committed  to  improving  soldier  perfor¬ 
mance  by  increasing  the  instructional  efficiency  of  training 
devices  and  their  built-in  instructional  subsystems.  Because 
computer-based  simulators  are  being  used  increasingly  for  mili¬ 
tary  training,  research  that  leads  to  improved  efficiency  of 
simulator-based  training  will  necessarily  yield  increased 
savings. 

ARI,  in  coordination  with  the  Project  Manager  for  Training 
Devices  (PM  TRADE)  and  the  University  of  Central  Florida's 
Institute  for  Simulation  and  Training,  has  begun  a  research 
program  to  examine  alternatives  to  conventional  simulator-based 
training.  The  research  reported  here  is  part  of  that  program;  it 
compares  the  effects  of  three  methods  for  teaching  psychomotor 
tasks:  (1)  easy-to-intermediate-to-dif f icult  ("crawl-walk-run”), 

(2)  difficult-only  practice,  and  (3)  mixed-difficulty  practice. 

The  work  described  in  this  report  is  part  of  ARI  Research 
Task  3104  and  was  performed  under  the  October  1991  Memorandum  of 
Understanding,  "Advanced  Technology  for  the  Design  of  Training 
Devices."  Plans  and  progress  were  briefed  to  PM  TR7U3E's  Chief 
Scientist,  Ronald  Hofer;  former  Product  Manager  (PM)  for  Close 
Combat  Training  Systems  (CCTS) ,  Colonel  Richard  L.  Knox;  and  the 
Deputy  PM  for  CCTS,  Phil  Sprinkle.  Copies  of  this  report  have 
been  delivered  to  PMs  and  other  managers  and  engineers  at  PM 
TRADE. 

The  results  of  this  research  raise  questions  about  conven¬ 
tional  device-based  training  methods  and  will  assist  PM  TRADE 
engineers,  contractors,  and  others  responsible  for  the  design  of 
instructional  subsystems  in  gunnery  and  other  training  devices. 


EDGAR  M.  JOHNSON 
Technical  Director 
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THE  EFFECTS  OF  EASY-TO-DIFFICULT,  DIFFICULT-ONLY,  AND  MIXED- 
DIFFICULTY  PRACTICE  ON  PERFORMANCE  OF  SIMULATED  GUNNERY  TASKS 


EXECUTIVE  SUMMARY 


Requirement: 

This  research  compares  the  effects  of  practicing  aasy-to- 
difficult,  difficult-only,  and  mixed-difficulty  training  exer¬ 
cises  on  students'  scores  on  a  test  of  mixed-difficulty,  simu¬ 
lated  gunnery  tasks. 


Procedure : 

Each  of  three  groups  of  20  male  undergraduates  practiced 
three  blocks  of  36  trials  under  one  of  three  conditions:  (1)  an 
easy-to-intermediate-to-difficult  progression  of  targets;  (2)  all 
difficult  targets;  or  (3)  a  randomly  ordered  mix  of  easy,  inter¬ 
mediate,  and  difficult  targets.  The  experimental  task  required 
tracking  and  shooting  moving  targets  with  TOPGUN,  which  is  a 
tank-gunnery  training  device.  All  three  groups  were  tested  on  36 
randomly  ordered,  easy,  intermediate,  and  difficult  targets  imme¬ 
diately  after  training.  Dependent  variables  were  target  hits, 
time  to  fire,  azimuth  error,  and  elevation  error. 


Findings: 

Results  indicate  that  (1)  learning  occurred  during  practice; 
(2)  differences  among  the  groups'  scores  on  the  test  of  mixed- 
difficulty  exercises  were  not  statistically  significant  for  any 
of  the  dependent  measures;  (3)  paid  ROTC  students  achieved  sig¬ 
nificantly  greater  hit  percentages  and  significantly  smaller 
aiming  errors  than  did  unpaid  non-ROTC  students,  who  received 
extra  course  credit  for  participation;  (4)  the  group  that  prac¬ 
ticed  the  easy-intermediate-difficult  progression  and  the  group 
that  practiced  randomly  mixed  targets  achieved  significantly 
greater  hit  percentages  with  easy  targets  during  testing  than  did 
the  group  that  practiced  only  difficult  exercises;  and  (5)  the 
group  that  practiced  the  easy-intermediate-diff icult  progression 
achieved  significantly  greater  hit  percentages  with  difficult 
targets  during  testing  than  did  the  group  that  practiced  randomly 
mixed  targets . 


vii 


Use  of  Findings: 


Several  hypotheses  and  post  facto  explanations  of  the 
results  were  offered;  their  tenability  can  be  established  in 
light  of  additional  research  results.  In  the  meantime,  one 
implication  for  military  training  remains  as  Wolfle  suggested  in 
1951:  Practice  should  vary  along  the  dimensions  and  range  of 
values  over  which  transfer  conditions  are  expected  to  vary. 

Varied  practice  seems  especially  germane  to  training  for  situa¬ 
tions  for  which  we  have  less  than  complete  knowledge  about  the 
characteristics  of  target  arrays  that  will  be  encountered  in 
combat.  Training  exercises  should  be  juxtaposed  in  ways  that  not 
only  promote  proficiency  on  each  exercise,  but  also  promote 
learning  of  the  discriminations  necessary  for  proficiency  in  the 
face  of  novelty  and  variety.  Exact  strategies  for  optimizing 
efficiencies  among  acquisition,  retention,  transfer,  and  general¬ 
ization,  however,  remain  to  be  discovered. 
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Introduction 

Nearly  all  modern  training  devices  incorporate 
instructional  subsystems,  which  include  software  and 
instructions  for  use  that  prescribe  an  instructional 
procedure,  a  sequence  of  exercises,  or  both.  The  built-in 
instructional  subsystems  are  more  likely  to  be  used  by  the 
instructor-operators  responsible  for  training  than  are 
alternative  procedures,  for  at  least  two  reasons;  (1) 
instructors'  belief  that  the  built-in  systems  offer 
instructional  sequences  that  are  sufficient  for  learning,  and 
(2)  the  inefficiency  of  alternatives,  whose  use  requires 
instructors'  programming  unique  sequences  of  exercises, 
essentially  creating  a  new  instructional  procedure  from 
scratch. 

Built-in  instructional  subsystems  do  not  always  reflect 
the  results  of  behavioral  science  research.  The  built-in 
instructional  subsystems  therefore  raise  interesting 
training-research  questions,  one  of  which  involves  the  use 
"crawl-walk-run"  methods — namely,  "Under  what  conditions  may 
learning  be  facilitated  by  the  use  of  alternatives  to  easy- 
to-intermediate-to-dif f icult  ( 'crawl -walk-run ' )  instructional 
sequences?"  Might  efficiencies  be  effected  in  some  cases, 
for  example,  by  practicing  exercises  of  mixed  difficulty  from 
the  outset  of  training,  or  by  practicing  only  difficult 
exercises? 

Adams  (1987),  Holding  (1962),  and  others  have  examined 
the  effects  of  practicing  exercises  in  three  sequences: 

1.  Easy  to  difficult. 

2.  Difficult  only. 

3.  Mixed  difficulty. 


Easy  to  Difficult 

The  Army's  "crawl-walk-run"  training  philosophy  (U.S. 

Army  Armor  Center,  1985)  exemplifies  trainers'  belief,  noted 
by  Jones  (1967),  that  initial  training  with  a  simplified 
version  of  a  task  will  lead  to  positive  transfer  to  more 
difficult  versions  of  the  task.  Skinner  (1953)  explained  the 
success  of  easy-to-diff icult  training  in  terms  of  the 
importaiiCe  of  early  success,  which  provides  continuous 
reinforcement  during  early  learning  of  new  tasks. 
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Empirical  support  for  easy-to-difficult  instructional 
sequences  is  available  from  research  on  part-task  training 
(Barton,  1921;  Seymour,  1954),  programmed  instruction 
(Pressey,  1927;  Holding,  1965),  and  adaptive  training  (Macrae 
&  Holding,  1965,  1966;  Poulton,  1974).  Although  much 
evidence  supports  the  efficacy  of  part-task  and  other  easy- 
to-difficult  training  methods,  the  support  is  not 
unequivocal.  Gagne  and  Briggs  (1979),  for  example, 
acknowledged  earlier  work  on  the  effectiveness  of  part-task 
training,  but  found  no  clear  advantage  for  practicing  whole 
or  part  skills  for  acquisition.  They  cited  Bilodeau's  (1966) 
contention  that  part-skill  practice  does  not  always  lead  to 
more  efficient  task  acquisition. 

In  summarizing  the  status  of  part-task  training  research, 
Adams  (1987)  concluded,  "The  most  bewildering  body  of 
literature  ...  is  the  field  of  part-whole  transfer 
.  .  .  .  sometimes  whole-task  practice  was  found  to  be  the 
best,  sometimes  part-task  practice,  with  no  clear  principles 
emerging"  (p.  45) . 


Difficult  Only 

Training  on  the  criterion  or  target  task  is  widely 
accepted  as  a  method  for  effective  transfer  (Ellis,  1969) . 
(One  might  argue  that  learning,  but  not  transfer,  can  be 
demonstrated  when  the  training  task  and  the  transfer  task  are 
the  same,  as  they  are  when  the  criterion  task  is  used  in 
practice.  [J.  D.  Hagman,  personal  communication,  December 
1991]).  That  acceptance  is  exemplified  in  the  Army's 
admonition  to  "train  as  we  will  fight"  (U.S.  Army  Training 
and  Doctrine  Command,  1990) . 

Designers  of  device-based  training  sometimes  strive  for 
"high-fidelity"  systems  in  the  belief  that  the  more  similar 
the  training  system  is  to  the  parent  system,  the  greater  the 
training  value  will  be.  That  belief  receives  some  support 
from  behavioral  science  research.  Thorndike  (1913) ,  for 
example,  espoused  the  existence  of  "identical  elements"  in 
practice  and  target  tasks  as  the  condition  for  optimal 
transfer.  Identical  elements  were  not  well  defined,  however, 
so  have  had  little  practical  effect  on  training  design. 

In  an  experiment  by  Baker,  Wylie,  and  Gagne  (1950) ,  five 
groups  of  31  subjects  tracked  an  irregularly  moving  target 
with  a  pointer  controlled  by  a  small  hand  crank.  The  rate  of 
crank  movement  required  to  track  the  target  was  varied  among 
groups  and  was  used  as  the  index  of  response  difficulty.  The 
amount  of  positive  transfer  was  found  to  depend  upon  the 
similarity  of  the  training  and  transfer  tasks;  the  greatest 
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amount  of  transfer  occurred  when  there  was  no  difference 
between  the  training  and  the  transfer  tasks. 

A  variation  on  practicing  the  criterion  is  to  train  on  a 
task  that  is  more  difficult  than  the  criterion.  Chief 
advocates  of  this  procedure  seem  to  be  sports  trainers. 
Competitive  swimmers,  for  example,  may  practice  while  wearing 
baggy  shirts  to  increase  drag,  and  basketball  players 
sometimes  practice  with  smaller-than-regulation  hoops 
(Ecklund,  1975) .  Expectations  are  that  athletes  who  perform 
adequately  under  unusually  demanding  conditions  will  perform 
exceptionally  under  criterion  conditions. 

Holding  (1962)  and  Wells  and  Hagman  (1989)  have 
hypothesized  that  initial  training  on  a  difficult  form  of  a 
task  may  lead  to  superior  transfer  or  retention  than 
practicing  easier  forms.  Their  thinking  is  that  transfer 
from  a  difficult  form  of  a  task  requires  no  new  learning,  but 
that  transfer  from  an  easy  form  requires  learning  new  "task 
elements."  Holding  (1962)  used  his  principle  of  inclusion  to 
explain  the  effectiveness  of  difficult-only  training  for  some 
tasks:  If  the  difficult  task  includes  the  easier  task,  then 
positive  transfer  will  result. 

Gagne  and  Briggs  (1979)  advocated  whole-part  learning,  in 
which  trainees  learn  the  total  task  sequence  before 
practicing  individual  parts,  under  some  conditions.  The 
whole-part  variation  on  difficult-to-easy  practice  also  was 
endorsed  by  Newell  (1981) ,  who  noted  that  as  a  general  rule 
skills  should  be  practiced  as  wholes;  only  when  whole-skill 
practice  is  prohibitively  difficult  should  part-skill 
practice  be  used,  and  then  only  with  meaningful  parts. 


Mixed  Difficulty 

Support  for  the  use  of  exercises  of  mixed  difficulty  is 
found  in  research  demonstrating  the  benefits  of  variety  in 
practice.  Wolfle  (1951),  for  example,  suggested  that 
practice  should  be  varied  along  the  dimensions  and  range  of 
values  over  which  transfer  conditions  are  expected  to  vary. 

He  criticized  military  training  during  World  War  II  on  the 
grounds  that  the  training  material  lacked  variety,  a 
deficiency  that  led  to  rapid  acquisition  during  training,  but 
resulted  in  poor  transfer.  Learning  did  not  generalize  from 
original  training  conditions  to  the  various  conditions 
encountered  on  the  job.  Wolfle  also  noted  that  variety  in 
practice  may  slow  acquisition,  but  that  benefits  would  be 
realized  in  the  range  of  conditions  to  which  learning  would 
transfer.  He  added  that  variety  in  practice  also  might 
counter  unwanted  effects  of  monotony. 
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Schmidt  and  Bjork  (in  preparation)  addressed  how  to 
maximize  learning  and  retention  for  situations  in  which 
several  different  tasks  must  be  learned  in  a  practice  session 
of  fixed  duration.  They  cited  research  by  Shea  and  Morgan 
(1979),  who  compared  the  effects  of  two  different  schedules 
of  practice  for  learning  three  motor  tasks.  In  one  schedule, 
the  practice  trials  for  each  of  the  three  tasks  were 
presented  in  a  logically  ordered  succession.  The  other 
schedule  comprised  an  order  of  practice  trials  that  was 
randomized  across  the  three  tasks.  The  randomized  schedule 
yielded  inferior  performance  during  initial  learning,  but 
better  retention,  than  did  practicing  the  orderly  succession 
of  trials  for  each  task.  Schmidt  and  Bjork  hypothesized  that 
the  random  schedule  forced  trainees  to  retrieve  and  organize 
task  information  on  each  trial,  and  that  practicing  retrieval 
led  to  better  retention. 


Variables  Affecting  the  Choice  of  Instructional  Method 

Some  researchers  have  attempted  to  organize  the 
considerations  discussed  above,  to  develop  guidance  for  using 
various  instructional  methods.  In  his  work  with  sequencing 
cognitive  instruction  Briggs  (1968),  for  example,  addressed 
task  interdependency:  If  task  elements  are  independent  of  one 
another,  then  sequencing  the  elements  for  instruction  is 
irrelevant;  if  elements  are  interdependent,  however,  then  we 
should  train  with  sequences  that  reflect  the  interdependency. 
Newell's  (1981)  comments  echoed  those  of  Briggs  (1968): 
Trainers  should  use  easy-to-difficult  training  procedures  if 
task  steps  are  interdependent. 

Task  type  may  influence  the  choice  of  sequence  for  easy 
and  difficult  practice.  Poulton  (1974)  advocated  identifying 
the  most  appropriate  forms  of  easy-to-dif f icult  and  fixed 
training  methods  for  a  particular  task  before  comparing  the 
two.  Both  training  methods  have  unique  advantages,  which  may 
become  evident  only  if  training  is  administered  in  particular 
ways  or  with  particular  tasks. 

Hamilton  (1964)  Indicated  that  several  task  and 
environmental  factors  influence  the  effectiveness  of 
instructional  method:  The  factors  include  response  mode, 
degree  of  prompting,  provision  of  feedback,  and  the  stability 
of  theme  sequence. 
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Rationale 


Theoretical  arguments  and  some  empirical  evidence  support 
the  hypothesis  that  training  efficiency  may  be  increased  by 
training  with  alternatives  to  easy-to-difficult  sequences. 
Training  with  e<  sy  or  moderate-difficulty  tasks  may  cause 
trainees  to  develop  habits  or  strategies  that  are  inefficient 
or  counterproductive  when  applied  to  more  difficult  tasks. 
Such  inappropriate  habits  could  be  costly  and  in  some  cases 
dangerous  for  persons  who  operate  weapons  systems.  Research 
to  date  has  not  yielded  practical  guidelines  for  determining 
the  conditions  under  which  easy-to-difficult,  difficult-only, 
and  mixed-difficulty  sequences  will  be  most  efficient. 

In  an  effort  to  derive  savings  that  might  be  realized  by 
using  alternatives  to  traditional  training  methods,  the  U.S. 
Army  Research  Institute,  under  sponsorship  by  the  U.S.  Army 
Project  Manager  for  Training  Devices,  initiated  a  program  of 
research  to  identify  the  conditions  under  which  the  use  of 
alternatives  to  traditional  training  methods  might  lead  to 
more  efficient  learning  of  psychomotor  tasks.  The  use  of 
alternatives  to  traditional  crawl -walk-run  methods  was 
examined  in  this  experiment. 


Purpose 

The  purpose  of  this  experiment  was  to  compare  the  effects 
of  practicing  easy-to-difficult,  difficult-only,  and  mixed- 
difficulty  training  sequences  on  students'  scores  on  a  test 
of  mixed-difficulty,  simulated  gunnery  tasks. 


Hypothesis 

We  hypothesized  that  students  trained  to  track  and  shoot 
simulated  tank  targets,  using  only  difficult  targets,  would 
achieve  test  scores  superior  to  the  scores  of  students  who 
trained  using  either  a  progressive,  easy-to-difficult  set  of 
targets  and  of  students  who  trained  using  blocks  of  trials 
comprising  targets  of  mixed  difficulty.  This  hypothesis  is 
consistent  with  Holding's  (1962)  principle  of  inclusion,  and 
was  based  on  our  belief  that,  because  all  practice  time  for 
the  difficult-only  group  would  be  devoted  to  difficult 
exercises,  the  difficult-only  group  would  have  much  higher 
scores  for  the  difficult  exercises,  which  when  combined  with 
their  scores  for  the  easy  and  intermediate  exercises,  would 
yield  higher  mean  scores  on  mixed-difficulty  exercises. 
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Method 


Subjects 

Sixty  men,  aged  19  through  29  years,  participated  in  this 
experiment.  Thirty  were  unpaid  undergraduates  from 
psychology  classes  at  the  University  of  Central  Florida 
(UCF) ,  who  received  extra  course  credit  for  participating; 
thirty  were  students  from  UCF  ROTC  units,  who  received  no 
extra  course  credit,  but  were  paid  $5.00  per  hour  for 
participating.  Ten  of  the  30  unpaid  students  and  10  of  the 
30  paid  students  were  randomly  assigned  to  each  of  three 
experimental  groups,  which  were  designated  PROG,  for 
progressive,  or  easy-to-intermediate-to-diff icult  training; 
DIFF,  for  difficult-only  training;  and  MIX,  for  mixed- 
difficulty  training.  Each  of  the  three  experimental  groups 
thus  comprised  20  students,  10  of  whom  were  paid  ROTC 
students,  and  10  of  whom  were  unpaid  non-ROTC  students. 


Apparatus 

The  TOPGUN  tank-gunnery  training  device  was  used  for 
tracking  and  shooting  simulated  tank  targets.  TOPGUN  uses 
computer-generated  imagery  to  display  representations  of 
targets  and  terrain.  Manual  controls,  similar  to  those  of 
tanks,  allow  trainees  to  track  and  shoot  targets  by 
manipulating  aiming  reticles  and  pressing  firing  buttons. 
Terrain  is  represented  by  narrow  colored  bands  extending 
horizontally  across  the  CRT  display. 

TOPGUN  exercises  of  various  difficulty  were  devised  for 
this  experiment  by  programming  variations  in  the  speeds  and 
ranges  of  targets  according  to  the  scheme  shown  in  Figure  1. 
(Speed  has  been  used  previously  to  manipulate  difficulty  in 
tracking  tasks  [Lincoln  &  Smith,  1951],  as  has  been  target 
range  [Green,  1955].)  Six  speed  values  were  crossed  with  six 
range  values.  The  nine  combinations  of  near  ranges  and  slow 
speeds  constituted  easy  exercises.  The  nine  combinations  of 
near  ranges  and  fast  speeds  constituted  intermediate- 
difficulty  exercises,  as  did  the  nine  combinations  of  far 
ranges  and  slow  speeds.  The  nine  combinations  of  far  ranges 
and  fast  speeds  constituted  difficult  exercises. 
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Figure  1.  Exercise  difficulty  defined  by  combinations  of 
target  ranges  and  speeds. 


Conduct  of  the  experiment  required  four  different  sets  of 
exercises,  each  set  comprising  36  exercises.  One  set, 
representing  a  mix  of  the  three  difficulty  categories  (easy, 
intermediate,  difficult)  comprised  each  of  the  36 
combinations  of  6  ranges  and  6  speeds.  Separate  sets  of 
exercises  also  were  formed,  which  comprised  easy-only, 
intermediate-only,  and  difficult-only  exercises.  The  easy- 
only  set  of  exercises,  for  example,  was  formed  with  four 
repetitions  of  each  of  the  nine  combinations  of  range  and 
speed  categorized  as  easy.  The  sequences  of  combinations  of 
ranges  and  speeds  were  randomized  within  and  across  each  set 
of  exercises. 

Each  exercise  contained  only  one  target,  which  was 
visible  to  the  student  for  15  sec.  Five-sec  breaks 
intervened  between  exercises. 

The  exercises  did  not  contain  natural  terrain  features 
such  as  boulders  or  cultural  objects  such  as  houses,  which 
would  have  concealed  targets  periodically.  The  lack  of 
terrain  features  and  cultural  objects,  and  the  color 
differences  between  targets  and  terrain  made  all  targets  easy 
to  detect.  Target  range  for  each  exercise  was  presented  on 
the  right  of  the  display,  unaccompanied  by  TOPGUN's  optional 
voice  announcement  of  target  range. 
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Procedure 


All  students  reported  to  the  trailer  that  housed  TOPGUN 
for  approximately  1.5  hour  of  training  and  testing.  Upon 
arrival,  each  completed  a  consent  form  (Appendix  A)  and  a 
background  information  form  (Appendix  B) ,  reporting  age, 
grade  level,  amount  of  video-game  experience,  and  avowing 
normal  color  vision.  Students  were  then  randomly  assigned  to 
the  PROG,  DIFF,  or  MIX  groups  and  were  treated  as  shown  in 
Table  1. 


Table  1 

Treatment  of  the  Compared  Groups  (n  =  20  per  group) 


Blocks  of  36  Trials 


Group _ 1 _ 2 _ 2 _ 4  (Test) 


PROG 

Easy 

Intermediate 

Difficult 

Mixed 

DIFF 

Difficult 

Difficult 

Difficult 

Mixed 

MIX 

Mixed 

Mixed 

Mixed 

Mixed 

Note.  The  mixed* target  blocks  comprised  9  easy,  18  intermediate, 
and  9  difficult  targets  presented  in  random  order. 


After  assigning  a  student  to  one  of  the  three  treatments, 
the  experimenter  administered  instructions,  which  included 
general  information  about  TOPGUN,  how  to  manipulate  the 
controls,  and  how  to  use  the  sight.  (We  used  the  gunner's 
auxiliary  sight  or  GAS,  whose  use  requires  applying  various 
leads  to  targets  depending  on  their  ranges  and  angles  of 
movement.)  The  experimenter  also  told  each  student  that  only 
one  round  was  to  be  fired  at  each  target  (see  TOPGUN 
Instructions,  Appendix  C;  and  Script,  Appendix  D) .  The 
student  then  began  the  first  of  three,  36-trial  training 
blocks,  the  last  of  which  was  followed  by  a  36-trial  test 
block.  Each  training  and  test  block  lasted  approximately  12 
min,  and  5-min  rest  periods  intervened  between  blocks. 

The  same  experimenter  ran  all  60  students.  He  provided 
feedback  about  the  items  listed  on  the  script  in  Appendix  D 
and  corrective  feedback  to  address  various  errors;  if  a 
student  consistently  overled  or  underled  targets,  for 
example,  the  experimenter  made  suggestions  for  adjusting 
lead.  Students  also  received,  in  addition  to  the  visual 
representations  of  hits  and  misses  by  TOPGUN 's  graphics 
system,  feedback  from  TOPGUN,  which  listed  summary 
performance  measures  on  its  screen.  These  summary  measures 
included  total  time  spent  practicing,  number  of  rounds  fired. 
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mean  times  and  rounds  used  to  destroy  targets,  and  total 
numbers  of  targets  destroyed.  Students  were  given  no 
information  about  how  their  scores  compared  to  the  scores  of 
other  students. 

Data  were  coded  in  three  steps:  (1)  compilation  of  scores 
by  TOPGUN's  performance  analysis  software;  (2)  transfer  of 
scores  into  Lotus  1-2-3  format,  to  extract  usable  information 
and  to  structure  data  files;  and  (3)  transfer  of  Lotus  data 
into  SPSS-PC+  format  for  statistical  analyses.  All  files 
were  inspected  for  missing  data  and  for  anomalies  due  to 
equipment  malfunctions. 

Data  were  analyzed  using  t-tests  comparing  Block  1  and 
Block  3  scores  to  determine  whether  learning  occurred  as  a 
function  of  treatments;  3x2  (group  x  ROTC/non-ROTC) 
analyses  of  variance  (ANOVA)  to  determine  whether  and  how 
scores  differed  as  functions  of  treatments  and  of  ROTC 
membership;  3x3  (group  x  difficulty  level)  ANOVA  to 
determine  whether  and  how  hit  percentages  and  firing  times 
differed  as  functions  of  treatments  and  of  exercise 
difficulty;  and  Peritz  multiple  comparisons  (Martin  and 
Toothaker,  1989)  to  determine  between  which  groups 
differences  occurred  in  cases  where  group  x  difficulty-level 
interactions  were  significant. 


Performance  Measures 

The  speed  and  accuracy  of  each  trial  were  measured  by 
time  to  fire  in  sec  and  by  azimuth  and  elevation  error  in 
milliradians  from  center  of  target  mass.  Timing  for  the 
time-to-fire  measure  began  when  automatic  slewing  of  the 
sight  stopped,  and  ended  when  the  student  pressed  his  firing 
button.  Hit  percentages,  the  number  of  hits  divided  by  the 
number  of  rounds  fired  in  each  training  and  test  block  (36  in 
all  cases) ,  also  were  calculated. 

TOPGUN  provides  a  choice  of  three  kinds  of  azimuth  and 
elevation  error  scores:  (1)  reticle-aim  scores,  which  are 
differences  in  milliradians  between  reticle  placement  and 
target  center  at  the  time  of  firing;  (2)  lead-error  scores, 
which  are  deviations  from  the  computer's  optimal-lead 
ballistic  solution;  and  (3)  point-of- impact  scores,  which  are 
the  round's  vertical  and  horizontal  displacement  in 
milliradians  from  the  target's  center  at  the  time  of  the 
round's  impact.  We  used  point-of-impact  scores  for  four 
reasons:  (l)  Reticle  aim  scores  are  not  appropriate  for  use 
with  the  GAS,  whose  use  requires  leading  targets;  (2)  earlier 
research  by  Turnage  and  Bliss  (1990)  showed  TOPGUN's  reticle 
aim  scores  to  be  inaccurate  and  unreliable;  (3)  pilot  testing 
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showed  TOPGUN's  lead-error  scores  to  be  inaccurate;  and  (4) 
point-of-impact  scores  provide  comparability  with  measures  of 
aiming  error  used  in  other  simulators,  such  as  the  Conduct- 
of-fire  Trainers  and  the  Videodisc  Gunnery  Simulator. 


Results 

A  summary  of  missing  data  is  in  Table  2,  where  we  see 
that  about  one-guarter  of  the  azimuth  and  elevation  scores 
from  the  three  training  blocks  were  missing.  Personal 
communication  with  the  manufacturer  of  TOPGUN  (M.  Koziewicz, 
June  1991)  indicated  that  the  scores  probably  were  missing 
because  of  problems  in  TOPGUN's  data-recording  software. 

Table  2 


Missing  Data  for  Each  Dependent  Variable  in  the  Training 
and  Test  Blocks 


Measure 

Blocks  1-: 

J  Test  Block  4 

Number 

% 

Number 

% 

Target  Hits  (%) 

0/180 

0.00 

0/60 

0.00 

Fire  Time 

(sec) 

56/6484 

0.86 

1/2161 

0.05 

Az .  Error 

(rad/1000) 

1627/6484 

25.10 

259/2161 

11.99 

11.99 

Means  based  on  available  scores  (without  interpolated 
values  for  missing  data)  are  shown  in  Figure  2  for  time  to 
fire,  in  Figure  3  for  hit  percentage,  and  in  Figures  4  and  5 
for  elevation  error  and  azimuth  error.  Standard  deviations 
associated  with  the  means  in  Figures  2,  3,  4,  and  5  are  in 
Appendix  E.  The  MIX  and  DIFF  groups'  curves  in  these  figures 
show  similar  patterns  for  all  four  dependent  measures:  Mean 
hit  percentages  increase,  while  mean  time  to  fire,  mean 
elevation  error,  and  mean  azimuth  error  decrease  across  the 
three  training  blocks.  Means  for  the  PROG  group  follow  a 
different  pattern:  Hit  percentages  decline  and  times  to  fire 
increase  over  the  three  training  blocks,  probably  because  the 
targets  in  Blocks  2  and  3  were  farther  and  moved  faster  than 
the  targets  in  the  preceding  block.  Block  4  hit  percentages 
are  greater  than  Block  3  hit  percentages  for  all  groups. 
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Mean  Targel  Kills  (%)  Mean  Time  to  Fire  in  Secxjiids 


Blocks  of  36  Trials 

Figura  2.  Mmr  time  to  fire  ae  a  function  of  practice  and 
teat  blocka  for  each  group. 


Blocks  of  36  Trials 

Figure  3.  Mean  hit  percentage  aa  a  function  of  practice 
and  teat  biocka  for  each  group. 
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ean  Azimuth  Error  in  Rad/10(X)  Mear^Elev^Uon  i'l 


Blocks  of  36  Trials 

Figure  4.  Mean  elevation  error  as  a  function  of  practice 
and  test  blocks  for  each  group. 


Figure  6.  Mean  azimuth  error  as  a  function  of  practice  and 
teat  blocks  for  each  group. 
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To  determine  whether  learning  occurred  with  practice,  we 
conducted  t- tests  on  the  differences  between  Block  1  and 
Block  3  scores  for  all  four  dependent  measures,  using  the  MIX 
group's  scores  and  the  DIFF  group's  scores,  but  not  the  PROG 
group's  scores.  The  reason  for  not  analyzing  the  differences 
between  the  PROG  group's  Block  1  and  Block  3  scores  was  that 
the  PROG  group's  Block  3  exercises  were  more  difficult  by 
design  than  their  Block  1  exercises.  The  results  of  the  t- 
tests  are  in  Tables  3  and  4 ,  where  we  see  that  mean  scores 
for  all  four  dependent  measures  are  significantly  different 
between  Blocks  1  and  3  for  both  the  MIX  and  DIFF  groups,  with 
all  differences  in  the  predicted  directions. 


Table  3 

Results  of  t-Tests  (two-tailed,  d.f.  =  19)  between 
First-block  and  Third-block  Scores  for  the  MIX  Group 


Measure 

Block  1 
Mean 

Block  3 
Mean 

t-Value 

Target  Hits  (%) 

47.22 

59.03 

4.27* 

Fire  Time  (sec) 

5.79 

4.73 

-5.08* 

Az  Er  (rad,/1000) 

1.60 

1.15 

-6.59* 

El  Er  frad/lOOO^ 

0.37 

0.28 

-7.17* 

*  sig.  .001 

Table  4 

Results  of  t-Tests  (two-tailed,  d.f.  =  19)  between 
First-block  and  Third-block  Scores  for  the  DIFF  Group 

Block  1  Block  3 

Measure _ Mean _ Mean _ t-Value 


Target  Hits  (%) 

28.75 

44.44 

6.67* 

Fire  Time  (sec) 

5.69 

4.85 

-4.40* 

Az  Er  (rad/1000) 

1.70 

1.22 

-7.97* 

El  Er  frad/1000) 

_ _ 

0.24 

-2.39* 

*  sig.  .001 


Four  3x2  (group  x  ROTC/non-ROTC)  analyses  of  variance 
were  performed  on  Block  4  test  scores  to  examine  the  effects 
of  the  three  treatments  and  of  ROTC  membership  on  each  of  the 
four  dependent  measures.  The  ANOVA  results  are  summarized  in 
Table  5.  In  the  "Group"  column  we  see  that  differences  due 
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to  the  three  treatments  were  not  significant  for  any  of  the 
four  dependent  measures. 


Table  5 

F-Ratios  from  Analyses  of  Variance  of  Block  4  Test  Scores 
as  Functions  of  Group  and  ROTC/Non-ROTC  Membership 


Measure 

GrouD 

ROTC/ 

Non-ROTC 

Group  X 

ROTC/ 

Non-ROTC 

Target  Hits  (%) 

1.29 

7.26* 

0.46 

Fire  Time 

(sec) 

0.49 

1.89 

0.24 

Az .  Error 

(rad/1000) 

0.89 

5.77* 

0.34 

El .  Error 

(rad/1000) 

2.13 

4.56* 

0.17 

*  2  <  .05  (group  d.f.  -  2,  ROTC/non-ROTC  d.f.  -  1,  group  x  ROTC/non- 
ROTC  d.f.  -  2,  total  d.f.  -  59). 


The  "ROTC/Non-ROTC”  column  of  Table  5  shows  F-ratios  for 
the  effects  of  ROTC  membership.  Differences  between  ROTC 
students'  and  non-ROTC  students'  hit-percentage,  azimuth- 
error,  and  elevation-error  scores  are  significant,  with  all 
differences  favoring  ROTC  membership.  The  difference  between 
ROTC  and  non-ROTC  students'  mean  time  to  fire  is  not 
significant. 

Interactions  between  group  assignments  (treatments)  and 
ROTC  membership  were  examined  to  determine  whether  treatments 
were  confounded  by  differences  between  ROTC  and  non-ROTC 
students'  scores.  Summaries  are  shown  in  the  "Group  x 
ROTC/Non  ROTC"  interaction  column  of  Table  5,  where  we  see 
that  none  of  the  interactions  between  group  assignment  and 
ROTC  membership  was  significant,  indicating  that  ROTC 
membership  did  not  confound  treatments. 


Discussion 

Our  hypothesis,  that  the  DIFF  group  would  achieve  better 
scores  on  the  Block  4  mixed-difficulty  test  than  the  PROG  and 
MIX  groups  would,  was  not  supported  by  the  results  of  this 
experiment.  Our  initial  inclination  to  suggest  that  the 
results  do  not  support  Holding's  (1962)  principle  of 
inclusion  was  dispelled  on  consideration  of  what  was  and  was 
not  included  in  the  DIFF  group's  treatment.  The  comrinations 
of  ranges  and  speeds  of  the  targets  used  in  the  difficult- 
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only  treatment  did  not  include  the  combinations  of  ranges  and 
speeds  of  the  easy  and  intermediate  targets  in  the  Block  4 
mixed-difficulty  test.  The  targets  in  the  difficult-only 
treatment  therefore  provided  stimuli  that  precluded  learning 
discriminations  among  easy,  intermediate,  and  difficult 
targets  —  discriminations  that  may  safely  be  assumed  to  be 
essential  for  achieving  high  scores  on  the  Block  4  mixed- 
difficulty  test,  which  required  applying  different  amounts  of 
lead  to  targets  depending  on  their  speeds  and  angles  of 
movement.  A  revised  interpretation  of  Holding's  principle  of 
inclusion  can  thus  be  made  in  terms,  not  of  the  speeds  and 
ranges  subsumed  by  the  speeds  and  ranges  of  the  most 
difficult  targets,  but  of  inclusion  of  opportunities  to 
practice  the  discriminations  required  for  successful 
performance  of  mixed-difficulty  exercises.  The  revised 
interpretation,  in  retrospect  more  parsimonious  than  our 
original,  and  which  accounts  for  what  is  learned  during 
practice,  would  have  led  to  predicting  that  (1)  the  DIFF 
group  would  achieve  the  worst  hit  percentages  and  aiming 
errors  on  the  Block  4  mixed-difficulty  test,  because  the  DIFF 
group  received  no  prior  opportunity  to  practice 
discriminating  among  easy,  intermediate,  and  difficult 
targets;  (2)  the  MIX  group  would  achieve  the  best  hit 
percentages  and  aiming  errors  on  the  Block  4  mixed-difficulty 
test,  because  the  MIX  group  practiced  discriminating  among 
easy,  intermediate,  and  difficult  targets  throughout 
training;  and  (3)  the  PROG  group  would  achieve  better  hit 
percentages  and  aiming  errors  than  the  DIFF  group's  but  worse 
than  the  MIX  group's  on  the  Block  4  mixed-difficulty  test, 
because  the  PROG  group's  learning  discriminations  among  easy, 
intermediate,  and  difficult  targets  depended  upon  remembering 
how  targets  and  leads  differed  among  the  three  training 
blocks.  Thus,  according  to  our  revised  interpretation  of 
Holding's  principle  of  inclusion,  the  ordering  of  the  three 
groups'  Block  4  hit-percentage  and  aiming-error  scores  should 
be  MIX  best,  PROG  next-best,  and  DIFF  worst.  An  exception  to 
that  order  should  occur  in  time  to  fire;  The  DIFF  group 
should  have  the  fastest  mean  firing  time  on  the  Block  4 
mixed-difficulty  test,  because  the  shooters  would  save  time 
by  not  making  the  discriminations  necessary  to  apply  various 
leads  before  firing  —  or  at  least  would  make  fewer 
discriminations  than  the  MIX  and  PROG  groups  would  make. 

A  similar  ordering  of  the  compared  groups'  hit 
percentages  and  aiming  errors  —  MIX  best,  PROG  next-best, 
DIFF  worst  —  results  from  predictions  based  on  the 
similarity  and  the  order  of  presentation  between  the  Block  4 
test  items  on  the  one  hand,  and  the  items  practiced  by  the 
three  groups  in  Blocks  l,  2,  and  3  on  the  other:  Similarity 
between  practice  items  and  test  items,  and  between  order  of 
presentation  (random  during  practice  and  random  during 
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testing)  was  greatest  for  the  MIX  group,  next  greatest  for 
the  PROG  group  (non-random  order  of  presentation) ,  and  least 
for  the  DIFF  group  (no  practice  with  easy  and  intermediate 
targets) .  Unlike  the  firing-time  prediction  from  the  revised 
principle  of  inclusion  presented  above,  the  prediction  from  a 
hypothesis  based  on  similarity  of  test  items  and  practice 
items  is  that  the  DIFF  group  would  have  slower  mean  firing 
times  than  the  MIX  and  DIFF  groups  on  the  test,  because  the 
test  items  were  more  similar  to  the  MIX  and  PROG  groups' 
practice  items  than  they  were  to  the  DIFF  group's  practice 
items. 

Notwithstanding  the  absence  of  statistically  significant 
between-group  differences  in  this  experiment,  our  revised 
principle-of-inclusion  hypothesis  and  the  test-practice 
similarity  hypotheses  presented  above  are  better  supported  by 
the  data  than  was  our  original  principle-of-inclusion 
hypothesis:  The  order  of  the  compared  groups'  scores  in  terms 
of  percentage  hits  and  elevation  error  (see  Appendix  E)  was 
MIX  best,  PROG  next-best,  DIFF  worst.  (An  exception  to  the 
revised  prediction  occurred  in  azimuth  error  because  of  a 
difference  of  two  one-hundredths  of  a  milliradian  between  the 
MIX  and  PROG  groups.)  In  terms  of  firing  time,  DIFF  was 
best,  MIX  next-best,  PROG  worst.  Predictions  from  both 
hypotheses,  one  based  on  our  revised  principle  of  inclusion 
and  the  other  based  on  similarity  between  test  items  and 
practice  items,  are  thus  supported  by  the  hit-percentage  and 
aiming-error  data.  As  for  firing  time,  however,  the  revised 
principle-of-inclusion  hypothesis  leads  to  predicting 
correctly  the  DIFF  group's  shooting  faster  than  the  MIX  and 
PROG  groups,  and  the  test-practice  similarity  hypothesis  does 
not. 


Although  no  differences  were  found  among  the  groups' 
scores  on  the  Block  4  mixed-difficulty  test,  the  possibility 
remained  that  the  three  treatments  differentially  affected 
each  group's  scores  on  the  easy,  intermediate,  and  difficult 
exercises  in  Block  4.  That  possibility  is  addressed  in 
Figures  6  and  7,  where  mean  hit  percentages  and  mean  firing 
times  are  shown  as  functions  of  the  three  levels  of  exercise 
difficulty  constituting  the  Block  4  test.  We  see  in  Figure  6 
that,  contrary  to  our  original  hypothesis,  the  DIFF  group's 
mean  hit  percentage  fell  between  the  mean  hit  percentages  of 
the  PROG  and  MIX  groups  on  the  difficult  exercises  in  the 
test.  Consistent  with  our  original  hypothesis,  however,  the 
DIFF  group  did  achieve  faster  mean  firing  times  than  did  the 
other  two  groups  on  the  difficult  exercises  in  the  Block  4 
test  (see  Figure  7) . 
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Figure  6.  Mean  hit  percentage  for  the  Block  4  test  as  a 
function  of  exercise  difficulty. 
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Figure  7.  Mean  time  to  fire  for  the  Block  4  test  as  a 
function  of  category  of  exercise  difficulty. 
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Three  x  three  (group  x  difficulty  level)  analyses  of 
variance  were  performed  on  the  data  in  Figures  6  and  7  to 
determine  the  separate  and  interactive  effects  of  treatments 
and  exercise  difficulty  on  Block  4  hit  percentages  and  times 
to  fire.  Summaries  of  the  analyses  are  in  Table  6,  where 
once  again  (cf.  Table  5)  we  see  no  significant  differences  as 
functions  of  group  assignment  for  either  hit  percentages  or 
firing  times.  Differences  in  both  hit  percentages  and  firing 
times  are  significant  as  functions  of  difficulty  level, 
however,  suggesting  that  our  definitions  of  exercise 
difficulty  in  terms  of  target  range  and  speed  were  valid. 


Table  6 

Analysis-of-variance  Summaries:  Target  Hits  and  Times  to  Fire 


as  Functions  of  Group 
Exercises 

and  Difficulty 

Levels  of 

Block  4  Test 

Source  fHit  %) 

SS 

DF 

MS 

F 

Group 

0.13 

2 

0.06 

1.20 

Difficulty  Level 

1.97 

2 

0.98 

61.24* 

Grouo  X  Diff.  Level 

0.33 

4 

0.08 

5.15* 

Source  fFire  Time  Sec) 

Group 

0.84 

2 

0.42 

0.11 

Difficulty  Level 

8.00 

2 

4.00 

16.52* 

Grouo  X  Diff.  Level 

1.74 

4 

0.43 

1.79 

*  sig.  .001 


Table  6  also  shows  a  significant  group  x  exercise-difficulty 
interaction  for  hit  percentage,  but  not  for  firing  time;  that 
is,  the  treatments  differentially  affected  the  compared 
groups'  hit  percentages,  but  not  their  firing  times,  on  easy, 
intermediate,  and  difficult  exercises. 

Results  of  Peritz  multiple  comparisons  (Martin  and 
Toothaker,  1989)  of  hit  rates  as  functions  of  group 
assignment  and  of  Block  4  target-difficulty  levels  are  shown 
in  Table  7,  where  we  see  that  three  of  the  nine  possible 
within-dif f iculty ,  pair-wise  comparisons  are  significant 
(with  a  =  .05).  The  two  significant  differences  between  hit 
percentages  on  easy  targets,  MIX  >  DIFF  and  PROG  >  DIFF,  are 
easy  to  explain:  Both  the  MIX  and  PROG  groups  had  practiced 
easy,  intermediate,  and  difficult  exercises  before  taking  the 
Block  4  mixed-difficulty  test.  The  MIX  and  PROG  groups 
therefore  had  opportunities  to  learn  the  discriminations 
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necessary  for  applying  different  leads  to  easy,  intermediate, 
and  difficult  targets.  Learning  those  discriminations 
allowed  the  MIX  and  PROG  groups  to  hit  a  greater  percentage 
of  the  easy  targets  on  the  Block  4  test  than  did  the  DIFF 
group,  which  had  no  prior  opportunity  to  practice  the 
discriminations . 


Table  7 

Results  of  Peritz  Multiple  Comparisons:  Target  Hits  as 
Functions  of  Group  and  Difficulty  Levels  of  Block  4  Test 
Exercises 


Comoarisons 

Dif f icultv 

of  Exercises 

in  Block  4: 

Easv 

Intermed . 

Difficult 

MIX  V.  DIFF 

MIX>DIFF* 

n.s. 

n.s. 

PROG  V.  DIFF 

PROG>DIFF* 

n.s. 

n.s. 

PROG  V.  MIX 

n.s. 

n.s. 

PROG>MIX* 

*a  -  .05 


The  PROG  group's  achieving  a  significantly  greater  hit 
percentage  than  the  MIX  group  on  the  difficult  exercises  in 
the  Block  4  test  is  more  difficult  to  explain.  A  hypothesis 
in  terms  of  amounts  and  recency  of  practice  does,  however, 
seem  tenable:  Before  taking  the  Block  4  mixed-difficulty 
test,  the  MIX  group  practiced  27  trials  against  difficult 
targets  —  nine  in  each  of  the  three  training  blocks  (see 
Table  1) .  Before  taking  the  Block  4  mixed-difficulty  test, 
the  PROG  group  practiced  36  trials  against  difficult  targets 
—  all  36  during  Block  3.  The  PROG  group  thus  practiced  a 
greater  number  of  difficult  exercises  more  recently  than  did 
the  MIX  group.  If  the  PROG  group  persevered  during  Block  4 
in  applying  the  leads  they  used  while  practicing  the  36 
exclusively  difficult  exercises  in  Block  3,  we  would  expect 
the  PROG  group  to  score  higher  against  the  difficult  targets 
(and  lower  against  intermediate  and  easy  targets)  in  Block  4 
than  the  MIX  group  would  (see  Figure  6) .  A  similar 
explanation  is  not  applicable  to  the  DIFF  group  (which 
practiced  the  same  number  of  difficult  exercises  as  the  PROG 
group  did  immediately  before  testing) ,  because  of  interfering 
effects  of  the  intermediate  and  easy  targets  encountered  for 
the  first  time  in  Block  4.  The  DIFF  group  thus  not  only 
failed  to  achieve  superior  overall  hit  percentages  on  the 
Block  4  test  of  mixed-difficulty  exercises,  but  also  failed 
to  achieve  superior  scores  for  the  difficult  test  exercises 
in  Block  4.  Those  results  are  especially  puzzling  on 
comparing  Block  3  hit  percentages  (see  Figure  3  and 
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Appendix  E)  for  the  PROG  and  DIFF  groups:  During  Block  3, 
both  the  PROG  group  and  the  DIFF  group  practiced  the  same  set 
of  difficult-only  exercises,  on  which  the  DIFF  group 
scored  45%  hits  and  the  PROG  group  42%;  that  is,  by  the  end 
of  Block  3  the  PROG  group  did  not  achieve  a  greater  hit 
percentage  than  did  the  DIFF  group.  On  Block  4,  however,  the 
DIFF  group  scored  50%  against  difficult  targets  (see  Figure 
6) ,  as  compared  to  45%  against  Block  3  difficult-only 
targets.  Thus,  even  though  the  DIFF  group  continued  to 
improve  their  hit  percentages  against  difficult  targets 
during  Block  4  testing,  the  possibilities  remain  that  (1) 
they  would  have  improved  even  more  in  the  absence  of  the 
intermediate  and  easy  targets'  interfering  effects  (J.  D. 
Hagman,  personal  communication,  December,  1991) ,  and  (2)  they 
may  have  surpassed  the  PROG  group  in  a  longer  test  of  mixed- 
difficulty  exercises.  That  second  possibility  is  related  to 
a  larger,  unresolved  issue  that  bears  on  the  results  of  this 
and  other  training  research.  The  issue  is  when  to  measure 
training  effects  and  relates  to  our  using  end-of-training 
test  scores  without  measures  of  transfer  and  retention  as 
dependent  measures.  That  training  methods  may  have 
differential  effects  on  initial  learning,  transfer,  and 
retention  is  well  known.  Gagne  (1954),  for  example, 
discussed  the  importance  of  measuring  effects  at  various 
intervals  after  training,  because  of  likely  changes  in  the 
direction  and  slope  of  transfer  and  generalization  functions 
over  time. 

In  a  recent  discussion  of  differential  training  effects 
on  initial  learning  and  retention.  Wells  and  Hagman  (1989) 
compared  the  merits  of  two  strategies  for  teaching  a  mixed 
set  of  tasks:  In  one  strategy,  blocking  similar  tasks 
together  during  practice  may  speed  initial  learning.  Doing 
so  corresponds  to  the  pattern  of  practice  used  in  the  present 
experiment  by  the  PROG  group.  The  second  strategy  is  to 
practice  dissimilar  tasks  mixed  within  blocks.  This 
strategy,  which  corresponds  to  the  treatment  of  our  MIX 
group,  was  hypothesized  by  Wells  and  Hagman  to  improve 
retention.  It  is  possible  that  each  of  the  three  strategies 
used  in  the  present  research  had  strengths  and  weaknesses 
such  that  none  proved  superior  to  the  others  only  in  terms  of 
test  scores  obtained  immediately  after  training.  That 
possibility  should  be  pursued  by  investigating  differential 
effects  of  various  training  methods  or  strategies  on  initial 
learning,  transfer,  retention,  and  generalization. 
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Summary  and  Implications 


Practicing  easy-to-diff icult,  difficult-only,  or  randomly 
ordered,  mixed-difficulty  training  exercises  effected  no 
significant  differences  in  target  hits,  times  to  fire,  and 
aiming  errors  on  a  test  of  randomly  ordered,  mixed-difficulty 
exercises  administered  immediately  after  training.  The  easy- 
to-difficult  and  the  mixed-difficulty  treatments  did, 
however,  result  in  significantly  greater  hit  percentages 
against  easy  targets  than  did  the  difficult-only  treatment; 
and  the  easy-to-diff icult  treatment  resulted  in  significantly 
greater  hit  percentages  against  difficult  targets  than  did 
the  mixed-difficulty  treatment. 

Several  hypotheses  and  post  facto  explanations  of  the 
results  were  offered;  their  tenability  can  be  established  in 
light  of  additional  research  results.  In  the  meantime  one 
implication  for  military  training  remains  as  Wolfle  suggested 
in  1951:  Practice  should  vary  along  the  dimensions  and  range 
of  values  over  which  transfer  conditions  are  expected  to 
vary.  Varied  practice  seems  especially  germane  to  training 
for  situations  in  which  we  have  less  than  complete  knowledge 
about  the  characteristics  of  target  arrays  that  will  be 
encountered  in  combat.  Training  exercises  should  be 
juxtaposed  in  ways  that  promote,  not  only  proficiency  on  each 
exercise,  but  also  learning  the  discriminations  necessary  for 
proficiency  in  the  face  of  novelty  and  variety.  Exact 
strategies  for  optimizing  efficiencies  among  acquisition, 
retention,  transfer,  and  generalization,  however,  remain  to 
be  discovered. 


21 


References 


Adams,  J.A.  (1987) .  Historical  review  and  appraisal  of 

research  on  the  learning,  retention,  and  transfer  of  human 
motor  skills.  Psychological  Bulletin.  101 .  41-74. 

Baker,  K.E.,  Wylie,  R.C.,  &  Gagne,  R.M.  (1950).  Transfer  of 
training  to  a  motor  skill  as  a  function  of  variation  in 
rate  of  response.  Journal  of  Experimental  Psychology.  40, 
721-732. 

Barton,  J.W.  (1921).  Small  vs.  larger  units  in  learning  the 
maze.  Journal  of  Experimental  Psychology.  4,  418-429. 

Bilodeau,  E.A.  (1966) .  Acouisition  of  skill.  New  York: 
Academic  Press. 

Briggs,  L.J.  (1968).  Seouencina  of  instruction  in  relation  to 
hierarchies  of  competence  (Final  report) .  Pittsburgh: 
American  Institutes  for  Research. 

Ecklund,  J.A.  (1975).  A  comparison  of  the  effect  of  variation 
in  task  difficulty  in  the  transfer  of  basketball  shooting 
accuracy  among  college,  senior  high,  and  junior  high 
females  (Doctoral  dissertation).  Eugene:  Microform 
Publications,  College  of  Health,  Physical  Education  and 
Recreation,  University  of  Oregon. 

Ellis,  H.C.  (1969) .  Transfer  and  retention.  In  M.H.  Marx 
(Ed.),  Learning:  Processes.  Vol.  1.  New  York:  MacMillan. 

Gagne,  R.M.  (1954) .  Training  devices  and  simulators:  Some 
research  issues.  American  Psychologist.  9,  95-107. 

Gagne,  R.M.  &  Briggs,  L.J.  (1979).  Principles  of 

instructional  design.  New  York;  Holt,  Rinehart,  and 
Winston. 

Green,  R.F.  (1955).  Transfer  of  skill  on  a  following  tracking 
task  as  a  function  of  task  difficulty  (target  size) . 
Journal  of  Psychology.  39,  355-370. 

Hamilton,  N.R.  (1964) .  Effects  of  logical  versus  random 

sequencing  of  items  in  an  autoinstructional  program  under 
two  conditions  of  covert  response.  Journal  of  Educational 
Psychology.  55 .  258-266. 

Holding,  D.H.  (1962).  Transfer  between  difficult  and  easy 
tasks.  British  Journal  of  Psychology.  53 .  397-407. 


23 


Holding,  D.H.  (1965) .  Principles  of  training.  Oxford: 

Pergamon  Press. 

Jones,  J.C.  (1967).  Learning.  New  York:  Harcourt,  Brace  & 
World. 

Lincoln,  R.S.,  &  Smith,  K.U.  (1951).  Transfer  of  training  in 
tracking  performance  at  different  target  speeds.  Journal 
of  Applied  Psychology.  35,  358-362. 

Macrae,  A.W. ,  &  Holding,  D.H.  (1965) .  Method  and  task  in 
motor  guidance.  Ergonomics .  8,  315-320. 

Macrae,  A.W.,  &  Holding,  D.H.  (1966).  Rate  and  force  of 

guidance  in  perceptual  motor  tasks  with  reversed  or  random 
spatial  correspondence.  Ergonomics .  9,  289-296. 

Martin,  S.A.  &  Toothaker,  L.E.  (1989).  PERITZ:  A  FORTRAN 

program  for  performing  multiple  comparisons  of  means  using 
the  Per it z  Q  method.  Behavior  research  methods . 
instruments .  &  computers .  21 ,  465-472. 

Newell,  K.M.  (1981).  Skill  learning.  In  D.H.  Holding  (Ed.), 
Learning  (pp.  203-226).  New  York:  Harcourt,  Brace  &  World. 

Poulton,  E.C.  (1974).  Tracking  skill  and  manual  control.  New 
York:  Academic  Press. 

Pressey,  S.L.  (1927).  A  machine  for  automatic  teaching  of 
drill  material.  In  A. A.  Lumsdaine  and  R.  Glaser  (Eds.), 
Teaching  machines  and  programmed  learning.  Washington: 
National  Education  Association  Department,  Audio-visual 
Instruction. 

Seymour,  W.D.  (1954).  Industrial  training  for  manual 
operations.  London:  Pitman. 

Shea,  B. ,  &  Morgan,  R.L.  (1979).  Contextual  interference 
effects  on  the  acquisition,  retention,  and  transfer  of  a 
motor  skill.  Journal  of  Experimental  Psychology:  Human 
Learning  and  Memory.  5,  179-187. 

Skinner,  B.F.  (1953).  Science  and  human  behavior.  New  York: 
Macmillan. 

Thorndike,  E.L.  (1913).  Educational  psychology,  ii.  The 
psychology  of  learning.  New  York:  Teachers  College. 

Turnage,  J.J.,  &  Bliss,  J.P.  (1990).  An  analysis  of  skill 
transfer  for  tank  gunnery  perfoinmance  using  TOPGUN.  VIGS. 
and  ICOFT  trainers  (Technical  Report  no.  916).  Alexandria, 


24 


VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences.  (AD  A231  156) 

U.S.  Army  Armor  Center  (1985).  Tank  platoon  mission  training 
Plan  (MTP  17-15-1) .  Fort  Knox:  Author. 

U.S.  Army  Training  &  Doctrine  Command  (1990).  Army  training 
2007;  Planning  for  army  training  (TRADOC  PAM  350-4) .  Fort 
Monroe:  Author. 

Wells,  R. ,  &  Ragman,  J.D.  (1989).  Training  procedures  for 
enhancing  reserve  component  learning,  retention,  and 
transfer  (Technical  Report  no.  860).  Alexandria,  VA;  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences.  (AD  A217  450) 

Wolfle,  D.  (1951).  Training.  In  S.S.  Stevens  (Ed.),  Handbook 
of  Experimental  Psychology  (pp.  1267-1286) .  New  York:  John 
Wiley  &  Sons. 


APPENDIX  A 


Consent  Form  and  Volunteer  Agreement,  Side  1 


I, _ having 

full  capacity  to  consent,  do  hereby  volunteer  to  participate  in 
research  entitled  Training  efficacy  as  a  function  of  instructional 
strategy  under  supervision  of  the  US  Army  Research  Institute.  The 
implications  of  my  voluntary  participation  and  the  nature, 
duration,  and  purpose  of  the  research,  and  the  method  and  means  by 
which  it  is  to  be  conducted  are  contained  on  the  reverse  side  of 
this  form.  I  have  been  given  an  opportunity  to  read  and  keep  a 
copy  of  this  Agreement  and  to  ask  questions  concerning  this 
research.  Any  such  questions  have  been  answered  to  my  full  and 
complete  satisfaction.  Should  any  further  questions  arise,  I  will 
be  able  to  contact  Don  Lampton  at  380-4368.  I  understand  that  I 
may  at  any  time  during  this  research  revoke  my  consent  and 
withdraw  from  the  test  without  prejudice,  but  I  will  not  be  paid 
unless  I  complete  2  full  hours. 


(Signature,  Date) 


I  was  present  during  the  explanations  referred  to  above  as 
well  as  the  volunteer's  opportunity  for  questions  and  hereby 
witness  his  signature. 


(Witness) 


(Date) 
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APPENDIX  A  (continued) 

Consent  Form  and  Volunteer  Agreement,  Side  2 

For  many  educational,  industrial  and  military  applications 
computer-based  training  simulations  provide  an  inexpensive  and 
safe  complement  to  training  with  operational  systems  and 
equipment.  This  experiment  is  part  of  a  project  to  develop  better 
methods  of  training  with  simulators.  The  purpose  of  this 
experiment  is  to  evaluate  training  effectiveness  as  a  function  of 
the  sequence  of  exercise  difficulty. 

You  will  be  asked  to  practice  an  aiming  task  using  a 
computer-based  training  simulator  which  was  developed  to  train 
tank  gunnery.  You  will  aim  and  fire  at  a  series  of  targets. 

After  each  shot  you  will  receive  an  indication  of  whether  you  hit 
or  missed.  The  task  involves  using  manual  controls  to  track  a 
target.  (The  device  display  and  the  required  task  are  somewhat 
similar  to  the  displays  and  tasks  used  for  many  video  games.) 

After  an  initial  practice  period  you  will  complete  4  sessions  of 
tracking  tasks.  Each  session  will  consist  of  approximately  20 
targets.  The  speed  and  range  of  targets  may  vary  across  sessions. 


Time 


Schedule: 


Activity 


00  -  15  minutes 
15  -  20  minutes 
20  -  40  minutes 
40  -  45  minutes 
45  -  65  minutes 
65  -  70  minutes 
70  -  90  minutes 
90  -  95  minutes 
95  -  115  minutes 
115  -  120  minutes 


orientation 
practice 
session  I 
break 
session  II 
break 

session  III 
break 

session  IV 
debriefing/payment 


The  risks  involved  are  those  associated  with  viewing  standard 
video  display  screens. 

You  are  free  to  terminate  participation  in  this  experiment  at 
any  time  without  bias.  However,  you  will  not  be  paid  unless  you 
complete  the  2  hour  session. 


Do  you  have  any  questions? 
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Background  Information 

The  purpose  of  this  questionnaire  is  to  collect  background 
information  on  participants  in  the  ARI  simulator  training 
research.  This  information  will  be  used  strictly  for  research 
purposes  only.  Please  complete  each  item  to  the  best  of  your 
ability.  Write  "N/A"  for  each  item  you  cannot  answer. 


1.  Name: _ 

Last  First  M.I. 

2.  Social  Security  Number: _ - _ - _ 

3.  Date  of  Birth: _ / _ / _ 

4.  Present  grade  classification  (Junior,  Senior, 

etc. ) _ 


5.  How  often  do  you  play  video  games  (circle  one)? 

A.  less  than  once  per  week 

B.  once  per  week 

C.  2-4  times  a  week 

D.  greater  than  4  times/week 

6.  Have  you  ever  been  diagnosed  as  color  blind/deficient? 
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APPENDIX  C 


Instructions 

Hello.  My  name  is  Jim  Bliss  with  the  Army  Research 
Institute.  Today  you  will  train  for  approximately  2  hours  on  the 
TOPGUN  tank  gunnery  trainer.  Please  seat  yourself  inside  the 
trainer.  You  are  seated  inside  a  trainer  for  the  gunner  position 
of  the  Ml  tank.  It  is  a  relatively  simple  device;  much  like  a 
video  game. 

In  front  of  you,  you  will  see  two  connected  handles.  These 
handles  move  the  gun  tube  up,  down,  and  side  to  side.  To  move  the 
aiming  crosshairs  side  to  side,  turn  the  handle  like  a  steering 
wheel.  To  move  the  crosshairs  up  or  down,  twist  the  handles 
accordingly  (demonstrate).  You  will  notice  a  set  of  buttons 
located  near  the  index  fingers'  position.  These  are  the  fire 
buttons  that  you  will  use  to  destroy  enemy  targets.  Most 
importantly,  in  order  for  any  buttons  or  handle  movement  controls 
to  operate,  THE  PALM  LEVERS  ON  THE  FRONT  OF  THE  HANDLES  MUST  BE 
ENGAGED! 1 ! 

During  the  following  engagements,  you  will  use  the  crosshairs 
shown  below.  When  the  automated  tank  commander  moves  you  to  the 
target,  you  must  immediately  estimate  and  implement  the  proper 
amount  of  horizontal  and  vertical  "lead"  required  to  hit  the 
target.  Horizontal  lead  means  you  must  aim  in  front  of  a  moving 
target;  Vertical  lead  means  you  roust  aim  above  distant  targets. 
You  will  have  help  with  vertical  lead;  when  a  target  is 
presented,  you  will  see  a  corresponding  range  figure  appear  in  the 
window.  The  numbers  on  the  side  of  the  reticle  correspond  to  the 
range  of  round  impact;  therefore,  you  must  position  the  reticle  so 
that  the  correct  range  is  on  the  target  when  you  fire. 

Example:  If  a  target  is  determined  to  be  1600  meters  away,  you 
should  put  the  center  of  the  line  beside  "16"  on  the  target  if  it 
is  stationary,  or  just  in  front  of  the  target  if  it  is  moving. 

Then  squeeze  the  fire  button. 


. 8 
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When  engaging  a  target,  the  sequence  of  activities  is  as 
follows:  1)  Squeeze  the  palm  levers  and  hold  them  down,  2) 

After  the  tank  commander  slews  you  toward  the  target,  estimate  the 
amount  of  horizontal  and  vertical  lead  required  to  effectively 
engage  the  target,  3)  Position  the  crosshairs  at  the  proper 
position  relative  to  the  target,  4)  Press  the  fire  button,  and, 
finally,  5)  Continue  monitoring  the  target  until  the  round  falls. 
Then,  quit  firing  and  wait  for  the  next  target.  Only  1  round  per 
target  is  permitted. 
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Target  Engagement  Script 

The  following  items  should  be  mentioned  to  trainees: 

1)  We'd  like  you  to  aim  for  the  very  center  of  the  target;  also, 
we'd  like  you  to  be  as  quick  as  possible.  Therefore,  speed  and 
accuracy  are  equally  important. 

2)  You'll  notice  that  the  range  figures  are  quite  exact;  so,  if 
the  range  to  a  target  is  1600  meters,  for  example,  you  need  to 
position  the  crosshairs  accordingly  or  you  will  miss. 

3)  Occasionally  a  round  may  look  like  it  hit  the  target,  but 
credit  is  not  given.  This  usually  means  that  the  round  landed 
just  beyond  the  target. 

4)  When  you  engage  targets  on  an  incline  or  decline,  you  must 
anticipate  their  vertical  movement;  so  if  a  target  is  at  1400 
meters  and  moving  uphill,  you  should  position  the  crosshairs  at 
1600  meters,  to  effectively  anticipate  the  target's  upward 
movement. 

5)  You'll  notice  that  the  range  figure  takes  a  split  second  to 
update  when  a  new  target  is  presented;  therefore,  you  should  make 
certain  of  the  range  before  firing. 


The  following  should  be  said  between  target  sets  (blocks). 

1)  Now,  I'd  like  you  to  lean  back  and  shut  your  eyes,  giving  them 
a  chance  to  rest  from  the  screen;  also,  move  your  hands  around, 
since  the  controls  are  somewhat  stiff. 

2)  OK,  the  basic  task  is  the  same  as  before.  Remember,  speed  and 
accuracy  are  equally  important.  When  you  feel  rested,  hit  the 
fire  button  to  begin. 
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APPENDIX  E 


Means  and  Standard  Deviations  (  ) 
for  the  Compared  Groups'  Scores 


Measure 

Group 

Block  1 

Block  2 

Block  3 

Block 

IFIRE 

PROG 

4.34 

4.76 

5.21 

4.71 

in  sec. 

(1.22) 

(1.32) 

(1.32) 

(1.05) 

DIFF 

5.68 

4.91 

4.85 

4.40 

(1.31) 

(1.44) 

(1.51) 

(1.18) 

MIX 

5.79 

5.16 

4.73 

4.66 

(1.38) 

(1.12) 

(1.00) 

(0.91) 

HIT 

PROG 

0.74 

0.50 

0.42 

0.63 

(.00) 

(0.10) 

(0.11) 

(0.14) 

(0.16) 

DIFF 

0.29 

0.42 

0.45 

0.59 

(0.11) 

(0.13) 

(0.13) 

(0.14) 

MIX 

0.47 

0.58 

0.59 

0.65 

(0.12) 

(0.10) 

(0.09) 

(0.09) 

AZ 

PROG 

0.92 

1.44 

1.20 

1.12 

Error  in 
Rad/1000 

(0.19) 

(0.37) 

(0.30) 

(0.29) 

DIFF 

1.69 

1.24 

1.22 

1.24 

(0.40) 

(0.35) 

(0.31) 

(0.37) 

MIX 

1.60 

1.19 

1.15 

1.14 

(0.33) 

(0.25) 

(0.23) 

(0.26) 

EL 

PROG 

0.34 

0.34 

0.22 

0.27 

Error  in 
Rad/1000 

(0.05) 

(0.05) 

(0.05) 

(0.07) 

DIFF 

0.29 

0.26 

0.24 

0.28 

(0.05) 

(0.07) 

(0.07) 

(0.04) 

MIX 

0.37 

0.29 

0.28 

0.24 

(0.05) 

(0.04) 

(0.05) 

(0.05) 

Note:  The  PROG  group  does  not  show  a  typical  learning  pattern,  because  target 
difficulty  increased  across  the  three  training  blocks. 
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